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ABSTRACT 

This  paper  is  in  response  to  a  growing  concern  about  the  adequacy 
of  present  measures  and  methodologies  in  depicting  air-to-ground  delivery 
system  accuracy.  An  accuracy  measure,  measurement  methodology  and 
performance  guarantee  based  on  cost-effectiveness  techniques  are 
developed.  The  measure  is  based  on  the  frequency  of  mission  success. 
The  methodology  is  based  on  Bayesian  techniques  using  a  multinomial 
distribution  to  represent  the  radial  miss  distance  pattern.  A  technique 
for  using  an  estimate  of  CEP  to  determine  the  prior  parameters  is 
developed.  Actual  and  simulated  impact  data  are  used  to  compare  the 
proposed  methodology  to  historically  accepted  and  other  recently  proposed 
techniques.  The  methods  of  cost-effectiveness  are  applied  to  guaranteeing 
an  air-to-ground  system  in  an  effort  toward  making  costs  more  controllable, 
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I.   INTRODUCTION 

A.  PURPOSE 

The  intent  of  this  thesis  is  threefold;  to  propose  a  measure  of 
effectiveness;  to  propose  a  method  to  determine  the  specific  effective- 
ness; and  to  propose  the  concept  of  a  cost-effectiveness  guarantee. 
All  three  are  proposed  with  respect  to  an  air-to-ground  (A/G)  weapons 
delivery  system  in  a  dive  maneuver.  However,  the  techniques  proposed, 
with  the  necessary  modifications,  appear  to  have  wider  applications. 
Some  of  the  development  is  incomplete  and  requires  further  investigation. 

B.  NEED  FOR  ACCURACY  MEASUREMENT 

Prior  to  developing  a  method  for  measuring  the  accuracy  of  an  A/G 
delivery  system,  it  is  important  to  understand  the  need  for  accuracy 
measurement.  Several  needs  are  discussed. 
1 .  Specification  Compliance 

In  the  Test  and  Evaluation  (T&E)  community  a  determination  of 
specification  compliance  is  usually  the  prime  motive  for  measuring 
accuracy.  The  accuracy  measure  and  occasionally  methodology  are  speci- 
fied in  the  applicable  contract  guarantees.  Confusion  can  be  introduced 
by  the  use  of  different  measurement  units  and  methodologies  in  different 

specifications. 

A  typical,  though  fictitious,  example  of  an  accuracy  specification 

is: 

"Exhibit  a  15  mil  circular  error  probable(XEP) ,  corrected  for 
aiming  error,  computed  in  the  plane  normal  to  the  line  of  sight 
from  the  release  point  when  delivering  Mk-76  practice  bombs 
at  a  release  airspeed  between  400  and  450  KIAS,  a  dive  angle 
„  between  40  and  50  degrees  and  a  release  slant  range  to  target 
less  than  10,000  feet." 

Testing  to  this  specification  would  probably  be  expensive. 


Also  inherent  in  the  specification  example  is  the  singular  usage 
of  the  measure  of  accuracy.  This  measure  in  itself  has  little  value 
outside  of  the  specification  context.  Therefore,  it  appears  that  a 
significant  amount  of  money  and  time  would  be  expended  with  negligible 
information  gain  beyond  specification  compliance. 

2.  Sortie  Predictions 

In  the  employment  of  an  A/G  system  there  is  a  need  for  a  differ- 
ent measure  of  accuracy.  Strike  planners  and  weaponeers  need  to  estimate 
the  probability  of  a  weapon  impacting  within  a  given  distance  of  a  target. 
(The  methodology  used  for  this  purpose  is  described  later.)  This  is  an 
important  need  as  it  is  required  throughout  the  lifetime  of  the  system. 

3,  System  Comparisons 

Often  a  need  arises  to  compare  a  system  with  some  other  system 
or  systems.  One  of  the  measures  of  effectiveness  that  should  be  used  for 
comparison  is  accuracy.  Thus,  a  need  arises  for  a  measure  of  accuracy, 
common  to  the  various  systems  being  compared.  An  identical  need  exists 
if  a  cost-effectiveness  analysis  is  to  be  conducted. 

Three  general  requirements  for  measurement  of  accuracy  have  been 
discussed.  As  presented,  each  of  these  requirements  uses  a  different 
measure  and  the  measures  need  not  be  compatibly  defined.  In  theory,  it 
could  occur  that  several  separate  determinations  of  accuracy  would  be 
required  for  a  single  system.  In  practice,  this  is  not  usual,  however, 
a  common  technique  would  be  beneficial  so  that  all  requirements  could 
use  the  same  measure. 

C.  HISTORICAL  MEASUREMENT  METHODS 

Historically  the  measurement  of  accuracy  of  nearly  all  types  of 
weapons  delivery  systems  has  involved  probability  distributions  and  the 
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parameters  that  define  these  distributions.  Several  terms  that  are 
commonly  used  in  weapons  accuracy  analysis  are  shown  in  Figure  1. 
1 .  Distributions 

The  normal  family  of  probability  distributions  has  long  been 
accepted  as  the  proper  family  for  the  distribution  of  weapon  impacts. 
Recently  this  concept  has  come  under  closer  scrutiny.  It  has  been 
suggested  that  a  normal  distribution  may  not  always  accurately  depict 
a  parent  distribution  of  impacts  and  other  families  such  as  the  Cauchy 
have  been  proposed  [1J. 

a.  Normal  and  Variations 

Many  different  variations  of  the  normal  distribution  have 
been  used  or  proposed  but  only  the  two  most  common  are  discussed. 

(1)  Bivariate  Normal .  The  bivariate  normal  is  the  more 
flexible  of  the  two  variations  discussed,  permitting  the  errors  in  range 
and  deflection  to  be  correlated.  Letting  x  denote  range  and  y  denote 
deflection,  the  density  function  is: 

(1)  f(x,y)  =  3 rexp{  -1  2  [(— ^-)2-2p  — £  — *  +  C— X)2]) 

2™xa  (1-p  )    2(1  -p  )   ax       c?x    a  a 

Often  the  range  and  deflection  errors  are  assumed  to  be  nearly  uncorre- 
cted so  that  p  is  approximately  zero.  A  rotation  of  the  coordinate 
system  can  also  be  used  to  eliminate  the  correlation  but  the  resulting 
variables  do  not  represent  true  range  and  deflection  errors.  The 
uncorrelated  density  reduces  to: 

(2)  f  U,y)  =  5^_  exp  <  4  E  (^)2  ♦  (^)2  ]  ) 


Range  Error   (x) 


Aircraft  Flight 

Path 


mpact 
Radial  Error   (r  ) 


Deflection  Error   (y  ) 


Target  Run-In  Line 


Figure    I. 

Depiction  in  the  Ground  Plane  of  terms 
commonly  used  in   Weapons  Accuracy 
Analysis. 
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The  parameters  are: 
y   -  mean  of  the  range  error  distribution 
u   -  mean  of  the  deflection  error  distribution 

y 

a        -  variance  of  the  range  error  distribution 
a   2  -  variance  of  the  deflection  error  distribution 
p  -  correlation  between  the  range  and  deflection  errors 

Standard  statistical  techniques  can  be  used  to  estimate  these  parameters 
from  observed  data. 

(2)  Circular  Normal .  The  more  commonly  used  normal  distri- 
bution is  the  circular  normal  in  which  it  is  assumed  that  the  range  and 
deflection  errors  are  independent  (uncorrelated) ,  have  mean  zero  and 
have  common  variance.  The  circular  normal  cumulative  function  where  r 
is  the  radial  miss  distance  is: 

(3)  F(r)  =  1  -  exp  {-  -^  }  ;   r  >  0 

2a 

Procedures  have  been  developed  for  use  when  ax  f   a 
[2,  p.  3  and  4];  i.e.,  the  distribution  is  an  elliptical  normal. 
An  axis  rotation  can  be  used  to  eliminate  the  correlation  between  range 

and  deflection. 

2 
An  estimator  for  the  a  parameter  is: 

N    2 
E  r. 

~2    i=l 

a       = 


2N 

The  circular  normal  has  computational  simplicity  over 
the  more  general  bivariate  normal  distribution,  but  the  assumptions  are 
quite  restrictive. 
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b.  Cauchy 

An  Air  Force  report  [1]  of  an  analysis  of  combat  impacts 
observed  in  Southeast  Asia  found  that  the  normal  family  did  not  yield  a 
good  distributional  fit  of  the  data.  One  of  the  distributions  that  was 
shown  to  closely  approximate  the  data  was  the  Cauchy. 

The  functional  forms  of  the  Cauchy  were  proposed; 

(4)  Rectangular  Cauchy   F(x,y)  =  -y  tan"  *~  tan"  -^ 

2 

(5)  Circular  Cauchy     F(r)  =  -  tan"1  (  -J «  )1/2 

B2  +  r2 

The  estimator  for  the  parameter  $  was  proposed  as: 
3  =  .455  •  (radial  miss  distance  of  the  median  impact) 

The  functional  forms  of  the  Cauchy  proposed  have  the 
advantage  of  simplicity  but  lack  mean  values. 

c.  Direct  Hits  Plus  Distribution 

The  Air  Force  study  [1]  pointed  out  that  several  impacts 
were  observed  to  be  directly  on  target  contrary  to  the  predictions  of 
a  continuous  distribution  function.  This  may  be  explained  by  considering 
the  method  of  measuring  the  miss  distances  (photographic),  the  physical 
size  of  the  target  and  some  ego  oriented,  psychological  attraction  of 
the  target  itself.  The  method  proposed  to  account  for  the  positive 
mass  at  the  target  was  to  introduce  a  proportion  of  direct  hits.  The 
distribution  function  then  becomes: 

(6)  G(x,y)  =  6  +  (1-6)  F(x,y)  in  the  bivariate  case,  or 

(7)  G(r)    =  5  +  (1-6)  F(r)  in  the  circular  case. 

The  parameter  6  could  be  estimated  from  prior  experience  or 
could  possibly  be  some  universal  constant  derived  from  many  different 
systems. 
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d.  Other  Distributions 

Literally  hundreds  of  different  distributions  such  as  the 
Weibul,  exponential  or  uniform  or  variations  of  distributions  such  as 
a  mixture  of  two  circular  normals  or  a  localized  normal  have  been 
proposed.  Two  interesting  facts  emerge  from  these  proposals.  First, 
as  the  predictive  ability  of  the  distributions  increases,  the  complexity 
of  the  functional  form  generally  increases.  This  usually  increases 
manipulation  difficulty.  Secondly,  nearly  all  of  the  distributions  or 
variations  proposed  are  continuous. 
2.  Measures 

There  are  several  commonly  accepted  measures  of  accuracy  which 
are  functionally  related  to  the  parameters  defining  a  unique  distribution 
in  an  assumed  family  of  impact  distributions.  These  measures  are 
graphically  portrayed  in  Figure  2. 

a.  CEP 

The  circular  error  probable  is  defined  as  the  radius  of  a 

circle  centered  at  the  target  (or  designated  point)  which  contains 

50  percent  of  the  observed  independent  impacts,  or,  the  radial  distance 

to  the  median  impact  [4,  p.  1-1,  1-2].  The  functional  relationship  of 

CEP  to  the  distribution  parameters  is  found  from  the  following  integral: 

CEP 
(8)  F(CEP)  =   /   f(r)  dr  =  .5 
0 

As  an  example,  the  CEP  using  the  circular  normal  distribution  is  derived 
as: 
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aircraft 
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heading 

▲ 


Figure   2. 

Depiction  of  Accuracy  Measures  in 
the  Ground  Plane. 
#  Individual  impact  points 
(  7  randomly  placed ) 
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.5    =     1-  exp  (-  £££-)  =     exp   (-  ^E-) 

ln(.5)   =  -^y- 
2a^ 

ln(2)       .     ™£ 

2a^ 

CEP     =     o\jl  ln(2)'        =     1.1774a 

b.  REP/DEP 

The  range  error  probable  and  deflection  error  probable  are 
similar  to  CEP.  The  REP  is  the  distance  from  the  target  (or  some  desig- 
nated point)  to  the  median  impact  in  range  [4, p.  1-2,1-3].  The  functional 
relationship  of  REP  to  the  distributional  parameters  is: 

REP 
(9)  F(REP)  -  F(-REP)  =  .5  =    /   f(x)  dx  where  f(x)  is  the 

-REP 

marginal  distribution  of  x  from  f(x,y) 

DEP  is  defined  similarly.  For  a  bivariate  normal  distribution, 

REP  =  .674a   and  DEP  =  .674a  . 
x  y 

It  should  be  noted  that  if  an  axis  rotation  was  used  to 
uncorrelate  the  range  errors  and  deflection  errors,  the  functional  form 
in  equation  (9)  is  no  longer  valid.  This  is  a  serious  disadvantage  of 
mathematically  uncorrelating  the  data. 

c .  MRE 

The  mean  radial  error  is  defined  as  the  mean  of  the  radial 
miss  distance  distribution.  It  is  derived  below  for  the  circular  normal 
distribution: 

r  r2 

fCr)     =    ~    exp  {-  i-y} 

a  2a 

co     2  2 

MRE     =     E[r]     =     /    *V  exp{-^T}dr 

0    *  2od 
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this  integral  is  solved  [5, p.  458]  as 
E[r]  =  —^a  =  1.253a  =  MRE 


Of  the  listed  measures,  the  MRE  is  probably  the  least  . 
frequently  used. 

d.  MP  I 

The  mean  point  of  impact  is  the  point  which  has  as  its  range 
and  deflection  coordinates  the  arithmetic  means  of  the  range  and 
deflection  coordinates  of  the  individual  impact  points  [4,  p.  1-3]. 
The  MPI  is  calculated  from  an  observed  sample  of  impacts  rather  than 
a  hypothetical  distribution.  In  many  analyses  (such  as  T&E)  [Refs.  18, 
20,23,  and  24],  the  MPI  is  used  as  the  center  of  the  observed  impact 
distribution  instead  of  the  target.  The  offset  of  the  MPI  from  the  target 
is  termed  a  system  bias  and  the  CEP,  REP,  DEP  and  MRE  are  calculated  with 
respect  to  the  MPI.  Obviously,  this  technique  would  not  be  useful  in 
weaponeering  applications. 

Analogous  to  the  MPI  are  the  mean  point  of  impact  in  range 
and  deflection  (MPIR,  MPID)  which  are  the  arithmetic  means  of  the  impacts 
in  range  and  deflection, 

e.  Other 

Three  other  definitions  given  in  the  Joint  Munitions  Effective- 
ness Manual  (JMEM)  [4,  p.  1-1,1-2]  are  of  interest  and  are  quoted. 

(1)  "Bombing  Error.  The  combination  of  all  errors  which 
cause  weapons  to  miss  the  target.  Included  are  ballistic,  aiming,  release 
and  aircraft  system  errors." 

(2)  "Ballistic  Dispersion.  The  variation  of  the  path  of  a 
weapon  which  is  attributed  to  physical  tolerances  in  the  weapon  dimensions 
and  aerodynamic  stability." 
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(3)  "Delivery  Accuracy.  The  measure  of  the  ability  of 
pilots  to  put  the  weapon  impact  pattern  center  (usually  MPI)  on  the 
target  or  aimpoint.  The  unit  of  measure  of  the  variation  in  placement 
of  the  pattern  center  may  be  a,  CEP,  or  REP  and  DEP.  It  is  these  measures 
that  are  used  in  predicting  the  results  of  future  weapon  releases  of  the 
same  type.  Delivery  accuracy  is  based  on  the  errors  in  aiming,  release 
and  aircraft  systems.  It  does  not  include  ballistic  errors." 
3.  Mils 

In  many  applications  the  accuracy  measures  are  expressed  in  mils 
(mil li radians)  perpendicular  to  the  line  of  sight  (LOS)  at  release  or 
some  other  appropriate  point  along  the  aircraft  flight  path.  The  geometry 
involved  in  the  computation  of  the  mil  is  presented  in  Figure  3.  A  mil 
is  usually  defined  as  the  angle  subtended  by  a  secant  line  of  one  foot 
length  at  a  radius  of  1000  feet  [4,  p.  4-2].  The  deflection  mil  error  (dm) 
is  related  to  the  deflection  foot  error  (df)  below  using  the  symbology 
of  Figure  3: 

1000  d^ 


m 


on  <im  =  — t-^ 

m      s 

Due  to  the  geometry,  the  deflection  error  in  the  ground  plane  is  identical 

to  the  deflection  error  in  the  scoring  plane,  thus  no  correction  is 

required.  The  range  mil  error  is  computed  by  the  following  equations: 

1000  r 
(12)  r   = where  r  can  be  closely  approximated 

(especially  if  s/r  is  large)  by 


(13)  rs  =  rf  cos 


It  seems  to  this  author  that  more  is  involved  than  just  the  ability 
of  the  pilot. 
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Aircraft  at  release  or  some  other  designated  point 
*  Horizontal 
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Figure  3. 
Geometry  of  the  Mil  Definition 

Depression  of  the  Los  from  the 

horizontal. 

Slant  range  to  the  target  in  feet. 

Range  miss  distance  in  the  ground 
plane  in  feet. 

Range  miss  distance  in  the  scoring 
plane  in  feet. 

Mil  error. 
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Theoretically,  the  use  of  mils  transforms  impact  data  taken 
from  various  release  slant  ranges  and  dive  angles  to  a  common  base. 
Traditionally,  separate  accuracy  measures  are  specified  for  each  given 
set  of  release  conditions;  i.e.,  slant  range,  flight  path  angle,  airspeed 
and  maneuver.  The  use  of  mils  permits  some  aggregation  of  accuracy  data 
over  release  conditions. 

D.  PROBLEM 

From  the  material  thus  far  presented  it  is  concluded  that  a  single, 
general  accuracy  measure  and  methodology  could  be  advantageous.  Some  of 
the  more  important  attributes  of  a  good  measure  and  methodology  are: 

a.  suitability  to  each  need  for  the  measurement  of  accuracy. 

b.  exactness  of  the  estimation  methodology. 

c.  minimal  amount  of  testing  required  to  achieve  the  desired 
estimation  exactness. 

d.  freedom  from  distributional  and  other  assumptions  restricting 
the  applicability. 

e.  independence  from  delivery  conditions. 

f.  mathematical  tractability  and  computational  simplicity 

1)  for  data  analysis. 

2)  for  weaponeering. 

3)  for  maintenance  of  capability  records. 

g.  adaptibility  to  new  systems. 

Even  the  few  attributes  listed  above  indicate  the  difficulty  of 
deriving  an  "optimal"  measure  and  measurement  methodology.  Additionally, 


Exactness  is  defined  as  the  minimum  error  throughout  this  thesis. 
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the  preferential  ordering  assigned  the  attributes  by  an  individual  will 

significantly  influence  his  judgement  of  the  "goodness"  of  a  particular 

i 

method. 

By  carefully  defining  the  problem  and  attempting  to  give  preference 
to  all  phases  of  the  problem,  a  methodology  has  been  developed.  This  is 
not  proposed  as  the  "optimal"  method  but  it  does  have  more  of  the  attri- 
butes listed  above  than  previous  methodologies  known  to  this  author. 
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II.  ACCURACY  MEASURE 

Prior  to  developing  a  methodology  for  determining  accuracy,  a  specific 
measure  of  accuracy  should  be  determined. 

A.  SYSTEM  FUNCTION 

The  JMEM  definition  of  delivery  accuracy  (quoted  earlier)  can  be 
paraphrased  as  the  placement  of  an  impact  pattern  center  on  a  designated 
point.  The  measure  of  accuracy  being  some  description  of  the  precision 
of  that  placement.  However,  accuracy  of  an  A/G  delivery  system  may  also 
be  thought  of  as  the  frequency  with  which  a  system  performs  its  assigned 
function  or  functions.  This  definition  differs  from  the  JMEM  in  that  it 
does  not  include  assumptions  about  an  impact  pattern. (it  is  distribution 
free)  and  it  gives  the  accuracy  measure  in  terms  of  the  mission.  An  A/G 
delivery  system  has  the  singular  function  or  mission  in  a  combat  role  of 
target  destruction. 

Two  important  results  can  now  be  discerned.  First,  the  singular 
function  of  target  destruction  greatly  simplifies  the  development  that 
follows.  Second,  a  basis  for  the  proposed  definition  of  delivery  accuracy 
has  been  reached.  This  basis  can  be  built  upon  to  derive  a  measure  and 
methodology  that  are  related  directly  to  the  system  mission. 

B.  MEASURE  OF  EFFECTIVENESS 

Based  on  a  mission  of  target  destruction,  the  accuracy  measure 
definition  is  the  frequency  with  which  target  destruction  is  achieved. 
An  interpretation  of  the  frequency  could  be  the  number  of  targets  destroyed 
per  sortie.  Note  that  this  is  the  inverse  of  a  common  weaponeering 
measure,  the  number  of  sorties  required  to  destroy  a  target.  The  averaging 
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over  sorties  is  not  essential.  It  could  have  been  taken  over  weapons 
expended,  attacks  or  other  quantities.  Sorties  was  chosen  because  it 
is  commonly  used  as  a  normalizing  quantity  in  aviation  terminology. 

It  may  seem  that  the  number  of  targets  destroyed  per  sortie  is  more 
a  measure  of  effectiveness  (MOE)  than  an  accuracy  measure.  In  many 
contexts,  the  MOE  may  be  a  function  of  an  accuracy  measure.  For  brevity, 
the  term  MOE  will  be  used  and  should  be  interpreted  as  meaning  both 
accuracy  measure  and  measure  of  effectiveness. 

The  MOE  chosen  (targets  destroyed  per  sortie)  closely  resembles  that 
used  by  the  Weapon  System  Effectiveness  Industry  Advisory  Committee 
(WSEIAC)  [6,  p.  24,25]  which  presents  some  recent  analyses  of  related 
problems. 

C.  WEAPONEERING  EQUATION 

The  MOE  chosen  can  be  mathematically  expressed  by  a  probability 
statement;  the  probability  that  the  target  is  destroyed  in  a  certain 
number  of  sorties.  This  is  symbolized  in  the  familiar  concept  of  the 
probability  of  failure  equals  the  product  of  the  probabilities  of  failure 
on  each  of  several  assumed  independent  trails  as: 

(14)  a  =  1  -  (1  -  p}n*  where; 

a  is  the  probability  that  the  target  is  destroyed. 

p  is  the  probability  of  target  destruction  in  a  single  attack. 

n  is  the  number  of  sorties 

l  is  the  number  of  attacks  per  sortie. 

Solving  for  the  rate  of  target  destruction  per  sortie  O/n)  yields: 
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net    i/n    a-ln(l-p) 
(15>    1/n  -   ln(l-a) 

Equation  (15)  will  be  called  the  "weaponeering  equation". 

For  convenience  in  this  thesis,  the  number  of  attacks  per  sortie  U) 
will  be  assumed  to  be  unity.  In  many  applications,  such  as  against 
heavily  defended  targets,  a   is  unity  by  policy.  However,  no  loss  of 
generality  or  mathematical  inconsistencies  are  imposed  due  to  taking  l 
to  be  unity. 

Superficially,  the  probability  of  target  destruction  in  a  single 
attack  (p)  may  appear  to  be  identical  to  the  historical  measure  of 
accuracy.  A  closer  examination  shows  p  to  be  more.  Inherently,  p 
depends  on  the  destructive  radius  of  the  weapon/target  combination  {rQ). 
However,  the  results  of  the  WSEIAC  analysis  [7, p.  22-33]  give  a  deeper 
insight  into  p.  The  WSEIAC  report  concluded  that  p  should  be  related  to 
the  availability,  dependability  and  capability  of  the  system  by  the 
following  equation. 

(16)  p  =  ATDC    where 

A   is  the  transpose  of  the  vector  of  probabilities  that  the 
system  is  in  some  state  the  start  of  the  mission. 

D   is  the  matrix  of  probabilities  that  the  system  is  in 
some  state  at  the  required  mission  time  conditioned  on 
the  state  of  the  system  at  the  start  of  the  mission. 

C   is  the  vector  of  capabilities  conditioned  on  the  system 
state. 

Appendix  C  gives  an  example  of  the  use  of  the  above  equation  to  enhance 
understanding  of  this  important  concept. 

Now  it  can  be  seen  that  C,  the  capabilities,  are  what  were  commonly 
referred  to  historically  as  the  measure  of  accuracy.  An  important 
distinction  exists  though  that  requires  additional  investigation. 
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D.  CAPABILITY 

As  defined,  the  capability  is  related  to  r  and  conditioned  on  the 
system  state.  No  mention  is  made  concerning  the  delivery  conditions. 
This  is  a  radical  departure  from  the  historical  method  of  tying  the 
accuracy  to  a  specific  set  of  delivery  conditions.  In  theory  and  in 
reality,  the  probability  of  placing  a  weapon  within  a  given  distance  of 
the  target  is  dependent  on  the  delivery  conditions.  However,  the 
experienced  attack  aviator  who  has  flown  in  combat  or  attempted  to 
achieve  a  specified  set  of  delivery  conditions  over  an  unfamiliar  target 
and  terrain  on  the  first  attack  will  readily  admit  the  difficulty 
involved.  The  presence  of  enemy  defenses  affects  all  delivery  parameters 
and  can  cause  large  deviations  from  programmed  dive  angle,  airspeed, 
release  altitude  and  run-in  heading.  Cloud  conditions  different  from 
those  predicted  may  dictate  last  minute  changes  in  dive  angle,  release 
altitude  and  run-in  heading.  Winds  not  anticipated  can  cause  dive  angle 
and  run-in  heading  to  change  during  the  attack.  In  close  formation  attacks 
each  aircraft  usually  achieves  a  different  set  of  delivery  conditions. 

In  older,  manual  delivery  systems,  the  sight  setting  used  was  based  on 
a  precise  set  of  delivery  conditions.  Thus,  a  great  importance  was 
imposed  on  the  achievement  of  these  prescribed  conditions.  Now,  sophis- 
ticated systems  continually  compute  the  predicted  weapon  impact  point 
and  automatically  release  the  weapon  when  some  designated  point  coincides 
with  the  predicted  impact  point.  The  pilot  using  this  system  is  free  to 
vary  his  delivery  conditions  "within  reason"  as  he  deems  appropriate. 


Dive  angle  and  release  altitude  define  the  slant  range  to  the 
target  which  is  an  influential  parameter  in  the  accuracy. 
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It  seems  reasonable  to  insist  that  the  capability  measure  also  be 
free  of  delivery  conditions. 

The  "within  reason"  phrase  cited  above  gives  qualitative  bounds  on 
the  delivery  conditions.  These  bounds  should  reflect  the  current 
operational  tactics,  and  can  be  expected  to  change  with  time.  For 
example,  in  the  "Vietnam  Era,"  the  bounds  might  have  been: 

a.  Dive  angle  25  to  60  degrees 

b.  Release  altitude  4000  to  7000  feet 

c.  Airspeed  400  to  600  KIAS 

d.  Run-in  heading  0  to  360  degrees 
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III.  ACCURACY  ESTIMATION  METHODOLOGY 

The  proposed  MOE  requires  the  estimation  of  p  =  A  DC.  The  estimation 
of  availability  and  dependability  (A  and  D)  has  been  well  documented  in 
numerous  reports  including  the  WSEIAC  [7].  A  method  of  estimating 
capability  (C),  as  used  to  estimate  p,  is  developed  herein. 

A.  DERIVATION 

The  detailed  derivation  is  presented  in  Appendix  A.  The  derivation 
of  the  estimator  for  C  follows  a  Bayesian  approach.  The  techniques  of 
the  Bayesian  approach  are  explained  by  DeGroot  [8],  Savage,  Raiffa, 
Schlaifer  and  many  others.  The  derivation  is  summarized  below  to  provide 
continuity. 

A  squared  error  loss  function  was  derived  as: 

a  *        2 

(17)  L  =  5-  [ln(l-p)-ln(l-p)]   where  p  is  the  estimator  of  p. 

(ln(l-a))2 
The  Bayes  estimator  was  found  to  be: 

(18)  D*  =  E[ln(l-p)]  (the  symbol  "  *  "  indicates  a  Bayes  estimator) 

Members  of  the  multinomial  family  of  distributions  were  chosen  as 
the  sampling  distributions  because  they  can  be  used  to  approximate  any 
distributional  shape.  The  probability  mass  function  of  the  multinomial 
is: 

(19)  f(xr...,xk|N,PlJ...>Pk)  =  51! p^i  ...  p^k 

xl!  ••'  xk! 


A  Bayes  estimator  is  the  one  which  minimizes  the  expected  loss 
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k 
where    N  =   ex. 
i=l  1 


1  =  z     p. 
i=l  1 


p.  >_    0    i=l,...,k 

and  x.  is  the  number  of  impacts  observed  in  the  i   interval. 

The  conjugate  family  for  the  multinomial  is  the  Dirichlet.  Its  density 

function  is: 

r(a-|+  ...  +  ak)    a^-1       a^-1 

(20)  f(pr...,pk|ar...,ak)  =  — — -  Pl     ...  Pk 

r(a-|)  -••  rCak) 


k 
where   1  =  s:  p. 
i=l  1 


pi  >  0    1=1,..., k 


a  ^  >   0    i=l,...,k 


Using  the  conjugate  family,  the  parameters  of  the  posterior  Dirichlet 
distribution  of  the  p.  are  a,  +  x,,...,  a.  +  x.  . 

With  an  assumption  and  an  approximation,  the  Bayes  estimator  for  C  was 

found  to  be 

m   a-  +  x.  k 

(21)  C*  =  i      — where  a   =   E  fa.  +  x.)   ,  and  m  is  the 

i=l    ao  °    i=l  1    1 

smallest  integer  greater  than  or  equal  to 
r  /interval  length  of  the  multinomial 
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It  was  found  that  the  prior  parameters  could  be  assigned  by  assuming  a 
prior  value  for  CEP  and  making  some  assumptions. 

Appendices  B,  C  and  D  delineate  the  proposed  procedure  using  an 
example  problem. 

B.  ASSUMPTIONS 

Several  assumptions  are  made  in  the  measure  and  measurement  methodology 
derivation.  This  section  will  present  a  complete  listing  of  all  the 
assumptions  and  corresponding  justifications. 

1 .  Independent  Trials 

Independent  trials  were  assumed  in  the  weaponeering  equation 
(section  II. C).  In  testing,  the  independence  can  be  achieved  by  using 
only  the  first  weapon  dropped  in  a  series  or  by  randomization  of  the 
delivery  maneuvers  and  parameters  so  that  no  two  consecutive  deliveries 
are  the  same.  For  prediction,  the  independence  assumption  is  conservative 
as  it  neglects  the  possibility  of  multipass  improvement. 

2.  Accuracy  Definition 

The  "frequency  with  which  a  system  performs  its  assigned  function(s)" 
definition  is  one  of  many  possible  definitions.  The  rationale  behind 
the  particular  choice  was  the  direct  link  to  the  mission  as  discussed  in 
section  II. A. 

3.  Accuracy  Measure 

The  "rate  of  target  destruction  per  sortie"  is,  again,  one  of  many 
possible  choices.  The  rationale  was  discussed  in  section  II. B. 

4.  Single  Attack  per  Sortie 

The  assumption  that  i  -   1  was  made  for  convenience  and  has  no 
effect  other  than  simplification.  Any  value  may  be  assigned  to  i   without 
changing  the  derivation. 
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5.  Independence  from  Delivery  Conditions 

The  specific  assumptions  and  an  extensive  discussion  of  the 
rationale  was  presented  in  section  II. D. 

6.  Squared  Error  Loss  Function 

The  rationale  behind  the  choice  of  the  quadratic  loss  function 
was  the  generality  and  tractibility  of  that  form.  The  implicit  assumption 
of  the  loss  being  equal  for  overestimates  and  underestimates  is  a  simplifi- 
cation. If  separate  treatment  is  preferred,  the  methodology  is  still 
correct  as  shown  in  Ref.  17,  p.  195-197. 

7.  Radial  Miss  Distance 

The  radial  errors  were  analyzed  vice  separate  treatment  of  the 
range  and  deflection  errors.  The  rationale  being  that  any  emphasis  placed 
on  a  particular  heading,  or  more  importantly,  heading  relative  to  some 
target  axis  may  lead  to  erroneous  conclusions.  Ideally,  use  of  the  proper 
run-in  heading  can  increase  the  accuracy  but  due  to  the  reasons  cited  in 
section  II. E,  this  will  often  result  in  an  overestimate  of  the  true 
accuracy. 

The  radial  measure  is  expressed  in  the  ground  plane.  The  advan- 
tages of  using  mils  in  the  scoring  plane  are  obvious,  especially  in  view 
of  the  varied  release  conditions.  Unfortunately,  targets  are  usually 
found  in  the  ground  plane  and  most  weapons  detonate  on  impact  with  the 
ground.  The  geometry  of  the  problem  will  also  show  that  weapons  designed 
to  detonate  at  a  fixed  altitude  yield  to  a  miss  distance  measured  in  a 
ground  plane  projected  horizontally  up  to  the  burst  height. 

8.  Perfect  Reliability 

The  assumption  that  A  D  =  1  was  made  for  simplicity.  Assuming 
otherwise  complicates  the  mathematics  but  does  not  alter  the  concept. 
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9.  CEP  as  Assigned  Prior 

Assigning  a  CEP  is  one  of  several  methods  of  assigning  the  ou  *s 
required  in  the  prior  distribution.  The  rationale  for  choosing  CEP  is 
given  in  Appendix  A  but  other  schemes  might  suffice. 

10.  Uniform  Distribution  of  pj 

The  justification  for  setting  all  the  E[p..]  equal,  up  to  the  CEP, 
is  based  on  the  resulting  mathematical  simplicity.  Other  assignments 
might  be  acceptable  but  might  also  add  to  the  difficulty  of  analysis. 

11.  Each  pj  Distributed  Symmetrically 

Assuming  each  p.  is  distributed  symmetrically  about  its  expected 
value  is  another  simplifying  step.  Assignment  of  specific  values  to 
V[p.]  is  possible  for  i=l,...,j  but  adds  another  subjective  decision  to 
the  analysis. 

Deletion  of  assumptions  4  and  6  through  11  does  not  affect  the 
methodology.  The  mathematics  become  more  cumbersome  and  the  estimator, 
D*,  may  differ,  but  the  concept  remains  unchanged. 

Assumptions  9  through  11  simplify  the  determination  of  the  o^'s  and 
as  will  be  shown  in  the  numerical  analysis  [section  IV),  yield  good 
results.  However,  changing  these  assumptions  does  not  alter  the  estimator. 

C.  APPLICABILITY 

Both  the  accuracy  measure  and  methodology  have  been  derived.  It  is 
worthwhile  now  to  reflect  back  to  the  listed  attributes  (section  I.D) 
and  comment  on  the  compliance  with  them. 
1 .  Suitability  to  Needs 

The  three  needs  cited  were  specification  compliance,  weaponeering, 
and  system  comparison.  The  suitability  of  the  selected  MOE  to  the  latter 
two  needs  is  apparent.  The  weaponeering  measure  was  used  as  the  MOE  and 
an  MOE  is  one  of  the  essential  elements  of  system  comparison. 
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The  sutiability  to  specification  compliance  is  not  obvious  and 
in  view  of  present  specifications  is  even  dubious.  In  section  V  it  will 
be  shown  that  specifications  can  be  couched  in  terms  of  cost-effectiveness 
with  many  attendant  advantages.  The  suitability  should  become  apparent 
in  that  context. 

2.  Exactness 

The  methodology  derivation  results  in  a  Bayes  estimator  of  the 
accuracy  measure.  The  Bayes  estimator  derived  is  a  sufficient  statistic 
[8,  p.  159]  which  means  that  no  more  information  relative  to  the  estimate 
can  be  garnered  from  the  data  [8,  p.  155].  The  use  of  Bayes  procedures 
also  allows  probability  statements  to  be  made  about  the  parameter  (p^ 

of  interest. 

The  numerical  analysis  (section  IV)  shows  that  the  proposed 
methodology  is  the  most  exact  of  the  several  techniques  compared. 

3.  Minimal  Testing 

The  numerical  analysis  (section  IV)  shows  that  an  average  error 
in  n1  of  approximately  three  sorties  per  target  destroyed  can  be  achieved 
with  100  data  points.  For  r  values  in  excess  of  40  feet,  the  corresponding 
error  is  less  than  two.  The  100  impacts  is  less  than  that  required  to 
achieve  comparable  exactness  with  the  other  techniques  evaluated. 

4.  Distribution  Free 

The  multinomial  density  permits  the  data  to  define  its  own  distri- 
butional shape  as  shown  in  Appendix  A. 

5.  Independent  of  Delivery  Parameters 

The  independence  from  delivery  conditions  was  discussed  in 
section  II. D. 


Hhe  reason  for  shifting  to  n  vice  1/n  is  explained  in  section  IV 
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6.  Tractibility  and  Simplicity 

The  use  of  a  conjugate  family  simplifies  the  data  analysis  as 
hown  in  Appendix  B.  The  weaponeering  can  be  accomplished  from  one 
table  and  one  graph  as  shown  in  Appendix  D.  As  the  number  of  data  points 
increases,  the  influence  of  the  prior  decreases.  Thus,  operational  units 
which  usually  collect  extensive  data,  need  only  maintain  the  total  number 
of  impacts  in  each  of  the  appropriate  intervals. 
7.  Adaptibility 

The  adaptability  is  highly  dependent  on  the  nature  of  the 
adaption  required.  No  quantitative  assessment  of  the  adaptability  can 
be  made  without  knowing  the  specific  adaptation  required. 
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IV.  NUMERICAL  ANALYSIS 

Numerical  analysis  was  used  to  compare  the  proposed  method  with 
other  historical  techniques.  The  Center  for  Naval  Analysis  graciously 
provided  extensive  A-7E  weapons  delivery  data.  There  were  1244  impacts 
delivered  from  system  dive  maneuvers  with  reasonable  release  conditions. 
These  data  were  used  to  make  a  comparison  with  real  world  data.  Simu- 
lation data  was  used  to  make  comparisons  based  on  various  known 
distributions  with  known  parameters.  A  sensitivity  analysis  of  the  CEP 
value  used  for  the  prior  was  also  conducted. 

The  quantity  used  for  comparison  was  the  inverse  of  the  accuracy 
measure  proposed.  The  magnitude  of  the  actual  minus  predicted  1/n  would 
be  inversely  related  to  the  magnitude  of  n  (  l/nn|n-n|  )  and  would  lose 
meaning  if  n  were  not  presented.  In  the  interests  of  security,  the 
value  of  n  will  not  be  presented.  So,  A  =  |n-n|  is  the  parameter 
compared.  (This  term,  A,  is  referred  to  as  the  comparator  in  the  figures.) 

A.  A-7E  DATA 

The  accuracy  parameters  derived  from  the  A-7E  data  are  not  given,  again, 
in  the  interests  of  security. 

The  true  distribution  of  impacts  was  assumed  to  be  that  defined  by  the 
data.  The  large  number  of  impacts  (1244)  lend  credibility  to  this 
assumption.  The  n  used  in  computing  A  was  derived  from  the  percent  of 
observed  impacts  within  the  appropriate  r  value. 

The  techniques  compared  to  the  proposed  methodology  were  based  on 
the  commonly  used  circular  normal  distribution  and  on  the  recently  proposed 
Cauchy  distribution  (see  section  I.C.l.b).  Both  the  regular  distributions 
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and  the  distribution  plus  a  percentage  of  direct  hits  were  used.  The 
percent  of  direct  hits  was  that  observed  in  the  data.  The  parameters 
necessary  for  the  comparison  distributions  were  derived  from  the  observed 
data  using  the  estimators  shown  in  Section  I.C.I. a  and  I.C.I .b . 

A  sample  size  of  100  randomly  selected  data  points  was  chosen  as 
being  representative  of  a  small  number  of  observations  for  most  purposes, 
yet,  a  readily  achievable  number  in  a  test  environment.  Fifty  runs  of 
100  impacts  each  were  conducted  sequentially  so  that  a  total  of  5000 
random  data  points  were  drawn  from  the  1244  impacts  available.  As  a  test 
of  the  randomness  of  the  samples,  the  mean  radial  error  for  each  of  the 
50  runs  were  compared  and  no  two  were  found  equal  when  rounded  to  the 
nearest  integer  value. 

The  comparisons  were  conducted  for  r  values  of  30  through  80  feet  in 
10  foot  increments.  The  interval  length  (w)  chosen  for  the  multinomial 
sampling  distribution  was  10  feet.  The  a  (probability  of  target  destruc- 
tion) was  chosen  to  be  0.9S.  The  CEP  chosen  for  the  Bayes  prior  was  of  a 
nominal  value  and  was  over  15  feet  different  from  the  value  observed  in 
the  1244  impacts. 

Figure  5  presents  a  plot  of  a  versus  r  for  the  different  measurement 
methodologies.  Qualitatively,  the  circular  normal  yields  the  worst  a 
values  and  the  Bayesian  method  the  best.  The  Bayesian  method  completely 
dominates  for  r  values  of  50  feet  and  greater.  All  the  estimates  approach 
a  common  value  at  r  =  70  feet.  An  interesting  point  is  that  the  Cauchy 
distribution  appears  to  be  a  better  estimator  than  the  Cauchy  plus  direct 
hits  distribution. 

The  average  a  values  for  r  values  equal  to  and  greater  than  30  feet 
are: 
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1.  Normal  plus  direct  3.3 

2.  Bayesian  3.4 

3.  Cauchy  3.6 

4.  Cauchy  plus  direct  4.8 

5.  Normal  7.9 

The  slight  advantage  of  the  normal  plus  direct  distribution  results  from 

the  large  difference  at  r  =  30  feet.  It  can  be  concluded  that  the  exact 

knowledge  of  the  percent  of  direct  hits  was  of  considerable  aid  to  the 

normal  plus  direct  distribution.  This  is  substantiated  by  the  fact  that 

the  A  value  for  the  normal  plus  direct  is  smaller  at  r  =  20  feet  than 

at  r  =30  feet.  Removing  the  r  value  of  30  feet  results  in  both  the 
e  e 

Bayesian  and  Cauchy  moving  ahead  of  the  normal  plus  direct. 

Though  not  presented  in  figure  5,  several  other  techniques  for  calcu- 
lating the  multinomial  parameters  (p.)  were  compared  to  the  Bayesian. 
The  maximum  likelihood  estimator  of  p.  =  x.  /  N  and  several  weighted 
averages  of  three  and  five  adjacent  intervals  (i.e., 

"1.2  Pj-2  *  Vl  Pj-1  *  wi  Pj  +  wi*l  Pj+1  *  V2  Pj+2  , 
1  wi-2  +  Vl  +  wi  +  wi+l  +  V2 

yielded  a  values  significantly  greater  than  those  resulting  from  the 
Bayesian  technique. 

B.  SIMULATION 

In  order  to  test  the  Bayesian  technique  across  a  wider  set  of  possible 
impact  distributions,  the  desired  distributions  were  simulated.  The 
programs  used  to  generate  the  distributions  were  those  presented  in 
Ref.  10  except  for  the  Cauchy.  The  Cauchy  was  generated  by  solving 
equation  (5)  for  r  while  using  a  random  number  generator  to  assign  values 
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between  0  and  1  to  F(r).  A  CEP  of  150  feet  and  three  percent  direct  hits 
were  used  in  the  generation  of  data  points.  The  distributions  simulated 
were  the  regular  and  the  regular  plus  direct  hit  forms  of  the  exponential, 
circular  normal,  circular  Cauchy  and  uniform  and  a  mixture  of  two 
circular  normal  distributions.  The  mixture  consisted  of  30  percent  with 
CEP  =  60  feet  and  70  percent  with  CEP  =  200  feet.  The  simulation 
routines  were  verified  by  plotting  and  by  comparing  the  generated 
statistics  to  the  input  values. 

The  same  techniques  were  used  for  comparison  as  with  the  A-7E  data 
(section  IV. A).  The  percent  of  direct  hits  was  assigned  the  known  value 
of  three  percent.  The  r  value  chosen  was  50  feet  and  the  Bayes  prior 
was  CEP  =  150  feet.  One  hundred  runs  of  100  samples  each  were  conducted. 

Figure  6  presents  the  A  value  for  each  of  the  techniques  and  each  of 
the  simulated  distributions.  The  Bayesian  technique  provided  the  best 
estimator  for  the  exponential,  uniform  and  mixed  normal  distributions. 
Surprisingly,  the  Bayesian  technique  also  provided  the  best  estimator  for 
the  Cauchy  plus  direct  hit  distribution  but  by  a  yery   small  margin. 

Table  I  presents  the  same  results  in  terms  of  the  ranking  of  each 
technique  for  each  distribution.  The  Bayesian  was  best  overall. 

To  ensure  that  the  test  was  not  biased  by  the  use  of  the  exponential 
and  uniform  distributions,  these  were  removed.  Table  I  also  presents 
these  results  and  the  Bayesian  technique  is  still  seen  to  be  the  best. 
Interestingly,  both  the  Cauchy  and  normal  plus  direct  surpass  the  Cauchy 
plus  direct  when  the  exponential  and  uniform  distributions  are  removed. 

It  can  be  concluded  that  the  Bayesian  technique  provides  the  most 
exact  estimator  of  the  techniques  tested  over  a  wide  range  of  possible 
impact  distributions. 
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C.  SENSITIVITY 

The  A-7E  data  was  used  to  test  the  sensitivity  of  the  CEP  value  used 

for  the  Bayesian  prior.  As  before,  50  runs  of  100  samples  each  were 

conducted.  The  r  value  used  was  50  feet.  CEP  values  from  120  to  180 
e 

feet  in  increments  of  10  feet  were  tested. 

Figure  7  presents  the  A  values  versus  the  CEP  values.  The  A  values 
range  from  2.1  to  3.1.  The  minimum  A  value  does  not  occur  at  the  true 
CEP.  The  A  values  at  r  =  50  feet,  for  three  of  the  other  techniques 
are  presented  for  comparison. 

It  can  be  concluded  that  the  Bayesian  technique  is  relatively  insensi- 
tive to  the  prior  CEP  value  chosen  when  100  data  points  are  available,  and 
the  prior  CEP  value  is  reasonably  close  to  the  true  CEP  value. 
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V.  COST-EFFECTIVENESS  GUARANTEE 

The  purpose  of  this  section  is  to  attempt  to  employ  the  methodology 
of  cost-effectiveness  analysis  in  specifying  and  evaluating  a  contract 
guarantee  for  an  A/G  delivery  system.  Cost-effectiveness  analysis  has 
been  defined  as  "a  method  for  studying  how  to  make  the  best  of  several 
choices.  Cost-effectiveness  is  always  used  in  relation  to  the  effective- 
ness of  alternative  systems,  organizations  or  activities."  [11,  p.  1]  It 
is  apparent  that  comparison  of  alternatives  is  the  core  of  cost- 
effectiveness  as  applied  to  choice  theory.  If  its  techniques  are  applied 
to  a  single  system,  caution  must  be  exercised. 

It  can  be  seen  from  section  I.B.I  that  present  specifications  some- 
times guarantee  accuracy  in  a  restrictive  sense  which  inhibits  both  the 
guarantee  and  the  evaluation  to  determine  specification  compliance.  It 
is  also  notable  that  most  guarantees  avoid  the  issue  of  costs  except  for 
penalty  values. 

The  cost-effectiveness  approach  has  proven  valuable  in  the  choice  of 
a  system  from  a  set  of  alternatives  and  could  prove  valuable  in  guarantee- 
ing a  particular  system.  Additionally,  the  cost-effectiveness  based 
guarantee  provides  a  stepping-stone  toward  controlling  cost  overruns. 

The  MOE  (1/n)  required  in  a  cost-effectiveness  analysis  has  been 
presented  in  section  II.B. 

A.  COSTS 

1 .  Types  of  Cost 

The  three  types  of  total  or  life  cycle  costs  are  research  and 
development  (R&D),  investment  and  operating  costs.  To  fully  account  for 
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the  "cost"  of  a  system,  these  must  be  expressed  as  "economic  costs"  or 
benefits  lost  due  to  the  use  of  the  resources  required  to  develop,  procure 
and  operate  a  particular  system.  Thus,  these  costs  include  hardware, 
manpower,  new  facilities,  supplies,  dollar,  etc.;  everything  directly 
related  to  the  decision  to  achieve  the  system  [12,  p.  25,66-67]. 

Care  must  be  taken  when  costing  an  A/G  delivery  system  that  is 
to  be  installed  in  an  airframe  so  that  the  costs  do  not  include  those  that 
are  incurred  by  the  airframe  independent  of  the  A/G  system  [13,  p.  5]. 
Also  the  concept  of  sunk  costs  [12,  p.  33]  must  be  considered  if  some  of 
the  components  of  the  system  are  currently  developed  or  procured.  These 
costs  should  not  be  included. 

Another  cost  that  is  sometimes  erroneously  included  in  computing 
system  costs  is  the  attrition  of  systems  due  to  combat  or  operational  type 
losses.  Once  the  system  is  developed  and  procured,  the  only  pertinent 
cost  is  the  operating  cost. 

The  cost  of  the  weapons  expended  to  achieve  the  MOE  are  not 
included  due  to  their  negligible  effect  compared  to  the  other  costs.  The 
MOE  qualitatively  reflects  the  increased  weapons  cost  of  an  inefficient 
system. 

The  methods  of  measuring  the  costs  are  well  detailed  by  Fisher 
[12]  and  many  other  authors  and  need  not  be  repeated  here.  However,  the 
total  cost  concept  has  serious  connotations  in  regard  to  a  guarantee  that 
will  be  fair  for  many  different  systems  and  needs  to  be  further  discussed. 
2.  Total  Cost 

Total  costs  represent  the  total  resource  impact  or  full  economic 
cost  of  the  system.  Necessarily,  the  magnitude  of  these  costs  is  highly 
dependent  on  the  number  of  systems  purchased  and  the  operating  lifetime 
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of  the  system.   (The  buy  size  and  lifetime  are  normally  estimated  during 
the  conceptual  phase  of  a  system  and  the  determination  of  these  values  is 
not  essential  to  this  thesis.)  If  these  total  costs  are  used  to  determine 
a  cost-effectiveness  guarantee,  systems  with  a  large  buy  and/or  a  long 
lifetime  will  be  unfairly  penalized  by  the  requirement  for  a  higher 
effectiveness  level  than  a  similar  (in  performance)  system  of  which  only 
a  few  are  purchased  and  have  a  shorter  lifetime.  The  logical  conclusion 
is  that  some  form  of  normalization  is  in  order. 

Varying  opinions  exist  as  to  whether  or  not  costs  should  be 
normalized  [6,  p.  40  and  11,  p.  29].  In  general,  normalization  tends  to 
hide  what  the  total  cost  is  and  in  the  usual  context  of  cost-effectiveness 
analysis  it  is  important  to  be  fully  aware  of  the  total  cost.  However,  in 
the  proposed  context,  the  lack  of  normalization  creates  the  inequity  shown 
above,  Thus,  normalization  is  considered  applicable  in  the  guarantee 
context. 

3,  Cost  Model 

Close  examination  of  the  three  incremental  costs  shows  that: 

a.  The  R&D  cost  is  nearly  independent  of  the  buy  size  and  system  lifetime. 
Therefore,  this  cost  can  be  used  with  no  normalization. 

b.  The  investment  cost  is  most  dependent  on  the  buy  size  and  it  seems 
natural  to  normalize  it  over  the  number  of  systems  resulting  in  units  of 
dollars  per  system. 

c.  The  operating  cost  is  dependent  on  both  lifetime  and  buy  size. 
Normalizing  over  both  yields  units  of  dollars  per  unit  of  time  per  system. 
This  will  be  expressed  as  an  annual  operating  cost  per  system  in  this  thesis 

Now,  there  are  three  different  costs  with  three  different  dimen- 

2     4 
sions.  The  cost-effectiveness  model  has  been  transformed  from  E  to  E 

(where  E  is  Euclidean  n-dimensional  space). 
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The  normalization  of  the  costs  in  the  cost  model  is  unique  to 
the  guarantee  context  and  these  costs  should  be  used  with  care  in  other 
contexts. 

B.  COST-EFFECTIVENESS  RELATIONSHIP 

The  various  techniques  that  could  possibly  be  used  to  determine  the 
functional  relationship  between  effectiveness  and  cost  are  detailed  in 
many  sources  such  as  Theil  [14]  and  Raiffa  and  Schlaifer  [17].  Prior  to 
the  use  of  any  of  these  techniques,  however,  a  data  base  is  required. 

1 .  Data  Base 

A  proper,  though  not  extensive,  data  base  exists  for  determining 
the  functional  relationship.  The  cost  data,  reliability  data,  and  weapon 
delivery  accuracy  data  for  present  and  past  systems  is  historical  and  can 
be  used  [15,  p.  11-13]. 

The  data  available  needs  to  be  transformed  into  a  form  compatible 
with  the  cost-effectiveness  framework  outlined  above.  The  availability, 
dependability  (based  on  a  standard  mission  time)  and  capability  are  used 
to  compute  1/n.  The  cost  data  needs  to  be  partitioned  in  the  appropriate 
accounts;  i.e.,  R&D,  investment  per  system,  and  annual  operating  per 
system.  Other  adjustments  to  the  data  may  be  necessary.  Examples  of 
these  are  well  detailed  in  a  RAND  report  [15,  p.  17-32]. 

2.  Technological  Dynamics 

With  the  data  transformed  into  the  proper  framework,  two  other 
aspects  requiring  attention  still  exist.  First,  is  the  fact  that  the 
technology  under  which  each  of  the  systems  was  produced  may  not  be  the 
same.  The  natural  assumption  is  that  the  technology  is  increasing 
chronologically  and  that  later  systems  are  more  effective.  Increased 
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technology  has  historically  carried  an  increased  price  tag.  This  is 
partially  what  the  cost-effectiveness  curve  is  representing.  However, 
the  possibility  of  cost  decreasing,  technological  break-throughs  is  real. 
If  during  the  determination  of  the  functional  relationship,  one  or  more 
data  points  appear  not  to  fit  the  others  then  these  must  be  studied 
carefully  to  see  if  such  a  break-through  did  exist.  When  this  is  the 
case,  a  new  cost-effectiveness  frontier  should  be  generated  based  on  the 
new  technology.  Generally,  omitting  the  data  points  representative  of 
the  old  technology  will  be  sufficient.  It  may  be  though,  that  there  are 
too  few  data  points  from  the  new  technology.  In  this  case,  the  shape  of 

the  frontier  could  be  determined  using  the  old  technology  and  the 

4 
"height"  (in  an  E  sense)  from  the  new  technology. 

3.  Inflation 

The  second  aspect  is  that  the  dollar  used  to  cost  a  particular 
system  is  not  the  same  dollar  used  to  cost  chronologically  future  or  prior 
systems.  This  is  not  serious  though  as  well  developed  techniques  exist 
[15,  p.  23-32]  to  account  for  the  time  dependent  value  of  the  dollar.  All 
that  needs  to  be  done  is  to  select  a  date  for  the  base  value  of  the  dollar 
and  transform  all  dollar  values  to  this  base  date.  It  may  be  desirable 
to  change  the  base  date  periodically  to  keep  it  fairly  close  to  the 
present. 

4.  Functional  Relationship  Model 

The  statistical  technique  used  to  determine  the  functional 
relationship  between  cost  and  effectiveness  needs  to  include  the  ability 
to  explicitly  state  the  uncertainty  incurred  in  the  predicted  form.  A 
Bayesian  regression  technique  might  be  reasonable,  due  to  the  ability  to 
make  probability  statements  concerning  the  estimated  parameters  but  no 
extensive  study  of  this  has  been  conducted  by  the  author. 
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The  successful  completion  of  the  above  steps  will  result  in 

4 

the  depiction  (mathematically)  of  a  hypersurface  in  E  that  represents 

a  cost-effectiveness  frontier. 

5.  Uncertainty  Considerations 

When  estimating  a  hypersurface  of  unknown  form  from  a  few  data 
points,  there  is  a  high  degree  of  uncertainty.  Though  it  would  be 
appealing  to  use  the  generated  hypersurface  for  the  guarantee,  it  might 
be  unfair.  Thus,  the  prediction  uncertainty  is  required.  The  direction 
of  the  prediction  error  is  also  uncertain  and  in  fairness  the  applicable 
lower  bound  of  the  prediction  error  interval  should  be  used. 

C.  GUARANTEE 

4 
The  cost-effectiveness  guarantee  is  in  E  and  cannot  be  depicted 

graphically.  However,  an  interpretation  of  the  concept  can  be  portrayed 

using  a  representative  cost  axis  as  shown  in  Figure  8. 

Adding  a  minimum  effectiveness  and  maximum  cost  to  the  guarantee 
defines  the  areas  as  shown  in  Figure  8  including  the  feasible  region. 
The  minimum  effectiveness  requirement  is  straight  forward  but  the  maximum 
cost  has  hidden  implications.  In  order  to  guarantee  a  maximum  total  cost, 
that  total  cost  must  be  apportioned  among  the  modified  incremental  costs. 
Thus,  though  specifying  a  maximum  total  cost  is  preferable,  it  may  be 
difficult.  The  alternative  is  to  specify  maximum  incremental  costs 
where  the  sum  is  not  the  desired  maximum  total  cost. 

The  guarantee  also  must  specify  the  mission  time  from  takeoff  to 
on  target  and  the  r  value.  If  these  values  are  altered,  the  entire  cost- 
effectiveness  frontier  must  be  regenerated.  Therefore,  these  values  must 
be  chosen  wisely. 
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Figure  8. 
Typical  Cost- Effectiveness  Guarantee. 
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1.  Penalty  Assessment 

The  historical  method  of  assessing  penalties  is  to  assign  a  dollar 
penalty  for  each  incremental  unit  outside  the  guaranteed  value.  In  theory 
this  penalty  assessment  seems  acceptable  but  in  practice  it  often  happens 
that  the  penalty  costs,  when  assessed,  are  charged  back  to  the  procuring 
agency  disguised  as  increased  overhead,  ground  support  equipment  or  other 
type  costs  adding  to  the  total  system  cost.  The  theory  of  a  cost- 
effectiveness  guarantee  would  eliminate  this  practice  but  the  method  of 
penalty  assessment  could  also  eliminate  it.  The  manufacturer  could  be 
required  to  meet  the  guarantee  by  reducing  cost  or  increasing  effectiveness 

2.  Guarantee  Currency 

A  periodic  review  of  the  guarantee  would  be  necessary  to  keep  it 
current.  A  logical  mechanism  for  ensuring  that  the  review  is  accomplished 
would  be  to  require  a  regeneration  of  the  cost-effectiveness  hypersurface 
each  time  new  actual  data  becomes  available  and  require  a  constructive 
review  each  time  the  guarantee  is  used.  The  minimum  effectiveness  and 
maximum  cost  portion  of  the  guarantee  could  be  unique  to  each  system  and 
these  parameters  could  be  defined  during  the  latter  type  review. 

3.  Guarantee  Parameters 

The  parameters  for  the  cost-effectiveness  guarantee  are  summarized 


as 


a.  Minimum  effectiveness 

b.  Maximum  cost  of 

1)  R&D 

2)  investment  per  system 

3)  annual  operating  per  system 

c.  Cost-effectiveness  hypersurface 

d.  Penalty  assessment 
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e.  Mission  time  from  launch  to  target 

f.  r  value 

e 

Other  parameters  peculiar  to  some  system  or  overlooked  herein  might 
also  require  specification. 
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APPENDIX  A 
DERIVATION  OF  THE  ACCURACY  ESTIMATORS 

The  detailed  derivation  of  the  accuracy  estimators,  based  on  Bayesian 
techniques,  is  presented  below. 

A.  LOSS  FUNCTION 

The  concept  of  a  Bayesian  estimator  is  minimization  of  the  expected 
loss;  thus,  a  loss  function  is  needed. 

A  loss  function  is  a  function  that  assigns  a  number  (called  the  loss) 
to  each  combination  of  decision  and  state  of  nature  [8,  p.  122].  In  the 
problem  at  hand,  the  state  of  nature  is  the  theoretic  number  of  targets 
destroyed  per  sortie  (1/n).  The  decision  is  the  estimate  of  1/n,  symbol- 
ized as  1/n.  Any  nonnegative  function  of  the  error  (1/n  -  1/n)  can  be 
used  as  a  loss  function  for  estimation  [8,  p.  226].  The  most  commonly 
used  are  absolute  value  of  the  difference  and  squared  error  (quadratic). 

DeGroot  shows  [8,  p.  227-228]  that  the  quadratic  loss  is  an  acceptable 
approximation  for  a  wide  range  of  nonnegative  loss  functions.  Principally 
because  of  this,  the  quadratic  loss  was  chosen.  An  implicit  assumption 
involved  in  this  choice  is  that  the  disutility  of  an  overestimate  is 
identical  to  that  of  an  underestimate. 

The  loss  function  is 

*  2 
(A.l)  L  =  a  (1/n  -  1/n)    where  a  is  some  nonnegative  constant. 

Recalling  equation  (15)  (from  section  II. C)  with  %   =  1  ,  the  loss  can  be 

expressed  as: 


(A. 2)     L     =     — ^ [ln(l-p)   -  ln(l-p)]        where  p   is  the  estimator  of  p 

llTO-a) 
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Letting  the  decision  be  D  =  In(l-p)  and  the  state  of  nature  be 

e  =  ln(l-p),  L  =  K(D-e)2  where  K  =  a/ln2(l-a).  DeGroot  shows  [8,  p.  228] 

4 

that,  for  this  loss  function,  the  Bayes  decision  against  any  given  distri- 
bution of  e  is: 

(A. 3)   D*  =  E[e] 


B,  SAMPLING  DISTRIBUTION 

One  of  the  attributes  listed  in  section  I.D  was  that  the  methodology 
be  distribution  free.  The  use  of  a  multinomial  distribution  to  depict 
the  impact  pattern  achieves  this  goal.  The  form  of  the  multinomial 
probability  mass  function: 

(A.4)   f(x1,.,.,xk|N,p1,...,Pj)  =  x  ,N'>X  j  P1Xl---pkXk 

K. 

k 

where   Z  x.  =  N 
i=l  1 


z  p .  =  1 
i=l  7 


p.j  >^  0     i=l,,..,k 

permits  the  cumulative  distribution  curve  to  take  nearly  any  shape  from 
convex  to  concave  and  many  conbinations  thereof. 

Due  to  the  desirable  independence  from  run-tn  heading  shown  in  section 
II. E,  the  radial  miss  distance  was  chosen  as  the  variable  for  the  density. 


nomi 


Figure  A.l  relates  the  geometry  of  the  impact  pattern  to  the  multi- 
al  density.  The  p.  are  the  probability  of  an  impact  in  the  i 


1L 

interval  and  the  x.  are  the  number  of  impacts  in  the  i   interval. 
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Figure   A.l 
Relationship  of  Multinomial  Density  to  Target  Geometry. 
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The  length  of  the  intervals  (w)  can  be  assigned  as  desirable.  They 
need  not  be  of  uniform  length,  though  it  is  assumed  here  that  all  but 
the  k   will  be.  The  k   interval  will  include  the  portion  of  the  real 
line  beyond  the  end  of  the  (k-1 )   interval.  It  would  seem  logical  to 
choose  an  interval  length  that  is  some  function  of  the  measurement 
accuracy  of  the  data  collection  method.  A  reasonable  choice  seems  to  be 
an  interval  length  equal  to  twice  the  measurement  error  in  the  data 
collection.  The  data  analyzed  for  this  thesis  was  collected  with  a 
measurement  error  of  ±5  feet  leading  to  w  =  10  feet. 

Another  consideration  is  the  number  of  intervals  (k).  The  choice  of 
k  should  be  based  on  the  number  of  data  points  available  for  analysis, 
the  interval  length  and  the  maximal  miss  distance  of  interest.  The  observed 
distribution  of  impacts  may  also  influence  the  choices  of  k  and  w.  In 
general,  a  study  of  the  system,  observed  data,  and  data  collection  method 
should  enable  one  to  assign  reasonable  values  to  k  and  w. 

C.  CONJUGATE  FAMILY 

The  use  of  a  conjugate  family  in  Bayesian  analysis  simplifies  the 
mathematical  manipulation  and  ensures  an  estimate  based  on  a  sufficient 
statistic  [8,  p.  159]. 

The  conjugate  family  for  the  multinomial  is  the  multivariate  Beta 

or  Dirichlet  distribution  [8,  p.  174],  The  functional  form  of  the 

Dirichlet  density  is: 

r(a,+  ...  +  a.  )    ,  i        i 
(A.5)    fCpr...,Pk|ar...,ak)   =  3 £-  P^1'  *'-Pkak 


rCa-j)  •••   (a  J 


k 

where    z  p.  =  1 
1=1  1 


p.j  >  0    t=l k 

a.  >  0    1sl,..,,k 
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The  expected  value  and  variance  are: 

a.  k 

(A.  6)   E[p.]  =  — *—   where  a  =     Z     a. 
1      ao  °   i=l  1 

a^Ca   -  a.) 
(A.7)   V[p.J  =  -V^ L_ 

DeGroot  shows  [8,  p.  174]  that  if  the  prior  parameters  are  a,  ,...,<*. 
and  x,,...,x.  are  the  number  of  observations  in  each  interval,  then  the 
posterior  parameters  are  a,+x, ,. . .  ,a.  +x.  . 

D.  ESTIMATORS 

It  was  stated  in  paragraph  A  (Loss  Function)  that  the  Bayes  estimator, 
D*,  was  equal  to  the  expected  value  of  6  =  In(l-p).  There  is  an  obvious 
difference  between  the  distribution  of  e  and  the  posterior  distribution 
of  the  vector  of  p-'s.  One  assumption  and  one  approximation  are  used  to 
derive  the  Bayes  estimators  for  the  p-'s. 

The  assumption  involves  the  relation  p  =  A  DC.  C  is  the  probability 
of  an  impact  within  r  of  the  target  given  some  state  of  the  system. 
Thus,  C  is  related  to  the  posterior  of  the  p.'s  by: 

m 
(A. 8)   C  =   Z  p.     where  m  is  the  smallest  integer  satisfying  the 
i=l  1 


Note 


inequality  m  >_  r  /w 

..i        m 

that   E[C]  =  E[  E  p.]  =   E  E[p.]  =   Z   1    1 

i=l  1     i=l   ]     i=l    ao 


Assuming  the  reliability  of  the  system  is  unity  (A  D  =  1)  permits  setting 
E[p]  =  E[C]. 
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The  approximation  involves  the  relation  e  =  ln(l-p).  For  reasonable 
values  of  r  ,  the  values  of  p  are  relatively  small  (less  than  .2).  For 
such  values  of  p,  ln(l-p)  can  be  approximated  by  p  with  a  maximum  error 
of  approximately  15  percent. 

The  approximation  p  =  In(l-p)  permits  setting  D*  =  E[e]  =  E[p]  =  E[C] 
Thus,  the  Bayes  estimators  for  the  p.  are: 

a.  +  x. 


(A. 9)   p. 


* 


1 i 

ao 


and 


m  a.  +  x. 
(A. 10)   p*  =  i 


i=l    ao 


E.  PRIOR  PARAMETERS 

For  the  subject  problem  several  schemes  for  assigning  the  prior 
parameters  were  possible.  The  prime  factor  dictating  the  choice  was  to 
require  a  minimal  number  of  parameter  values  to  be  assigned. 

One  of  the  most  widely  used  and  well  known  accuracy  parameters  is  the 
CEP.  The  CEP  is  also  one  of  the  easiest  parameters  to  assign  a  priori. 
The  assignment  can  be  based  on  system  simulation,  system  design  or  some 
other  method  tempered  by  experience.  The  procedure  for  using  CEP  as  a 
single  assignment  parameter,  as  discussed  below,  is  Ad  Hoc.  Several 
assumptions  are  made,  the  justification  of  which  is  made  in  section  III.B, 

In  terms  of  the  p.'s,  the  CEP  is  defined  as: 


J 
.5  =   E  p.    where  j  =  CEP/w. 

1-1  1 


Assuming  E[p.]  are  equal  for  i  =  l,...,j  and  a  uniform  length  (w)  for 
intervals  1 j; 
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E[p.j]  =  .5/j  ;  i  =  l,...,j 

and  from  equation  (A. 6),  the  a.  are  equal  for  i  =  l,...,j.  Allowing  the 
k   interval  to  be  the  remainder  of  the  distribution,  E[pk]  =  .5. 
From  equation  (A. 6)  and  the  discussion  above, 

J 

ao  =  if1  ai  +  ak  =  jai  +  ak 


and    E[pj]  = 


ai 


ja.  +  aR 


thus, 


"S-  (ja_.  +  a,  )       i  -  1  ,. .  .  ,j 


and 


j  VJ"i   "k' 


tk  =   .5  (jet.  +  ak) 


These  equations  are  solved  to  find 

a.  =  ja.    i=l,...,j    as  might  have  been  expected. 

Attempting  to  use  the  derived  values  and  assigning  values  to  the 
variances  (equation  (A. 7))  leads  to  an  inconsistent  system  of  equations. 
This  can  be  resolved  by  assuming  that  each  p.  is  distributed  symmetrically 
about  its  expected  value.  Then  the  variance  of  the  p.  for  i=l,...,j  can 
be  set  such  that  95  percent  of  the  probable  p.  will  be  nonnegative.  It 
follows  that: 

V[Pi]  =  (E[Pi]/1.64)2 

For  i=l  ,...,j; 

VCp^  =  (1/3. 28j)2 
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Solving  for  the  a.  yields; 


(A.n)         -     =   LM-z^l      .^^ 
1  j 


and 


(A.  12)  «k     =     jc^.     =     2.69J   ~   .845 


It  has  been  shown,  with  several  assumptions,  that  the  prior  parameters 
can  be  derived  given  the  prior  CEP. 
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APPENDIX  B 
SAMPLE  DERIVATION  OF  p. 

The  procedures  for  using  the  Bayesian  technique  suggested  in  this 
thesis  are  illustrated  by  a  simple,  fictitious  example  involving  a  sample 
of  10  impacts.  Suppose  the  observed  radial  miss  distances  (in  feet)  for 
dive  maneuvers  in  the  prime  mode  are: 

7,  92,  56,  37,  23,  6,  88,  75,  29,  41 

An  interval   length  of  10  feet  and  prior  CEP  of  50  feet  will   be  used. 

j  =  CEP/w     =     50/10     =     5  thus     k  =  6 

.  2.69(5)  -  .845     =     ^2  i=l,...,5 

1  5 


ct6  =  12.61 


The  values  of  interest  can  be  tabulated  as  follows: 


i 

iiilci  va  i 

endpoints 

!i 

x.* 
l 

ai   +  Xi 

p.** 

1 

0-10 

2.52 

2 

4.52 

.13 

2 

10-20 

2.52 

0 

2.52 

.07 

3 

20-30 

2.52 

2 

4.52 

.13 

4 

30-40 

2.52 

1 

3.52 

.10 

5 

40-50 

2.52 

1 

3.52 

.10 

6 

50-°° 

12.61 

4 

16.61 

.47 

*  x.  is  the  number  of  impacts  observed  in  the  i   interval 

a,.  +  x.         6 
**  Pi 


1^  ;  a0-  z  C-!  +x.)  . 
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APPENDIX  C 
EXAMPLE  DERIVATION  OF  p 

The  equation  p  =  A  DC  given  in  the  WSEIAC  report  [7]  is  explained 
by  the  following  fictitious  example  using  the  C  vector  component  as 
derived  in  Appendix  B. 

Consider  a  system  which  has  a  prime  mode  and  one  degraded  mode.  The 
possible  system  states  are  defined  as; 
A  -  prime  mode 
B  -  degraded  mode 
C  -  inoperative 

Suppose  the  availabilities  (probability  that  the  system  is  in  some 
state  at  the  beginning  of  the  mission)  are: 
P{A}  =  1/2 
P{B}  =  1/4 
P(C}  =  1/4 
Thus,  AT  =  (1/2,1/4,1/4). 

Assume  the  system  cannot  be  repaired  in  flight.  The  dependabilities 
(probability  that  the  system  is  in  some  state  at  a  specified  time  after 
takeoff  given  the  system  state  at  the  beginning  of  the  mission)  are 
assumed  to  be: 

Given  state  A  at  the  beginning  of  the  mission-, 
P{A}  =  1/2 
P{B}  =  1/4 
P{C}  =  1/4 
Given  state  B  at  the  beginning  of  the  mission; 
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P{A>  *  0 

P{B}  =  3/4 

P{C}  =  1/4 

Given  state  C  at  the  beginning  of  the  mission; 

P{A}  =  P{B}  =  0 

P{C}  =  1 


Thus,  D  = 


1/2  1/4  1/4 
0  3/4  1/4 
0    0    1 


and  AD  =  (1/4,5/16,7/16) 


The  A  D  is  constant  for  a  constant  mission  time  and  in  most  systems 
will  probably  remain  fairly  constant  over  various  mission  times.  There- 
fore, in  the  weaponeering  usage  this  could  be  accepted  as  a  set  of  values 
unique  to  each  A/G  delivery  system.  Then,  only  the  final  calculation 
(shown  next)  need  be  performed  for  various  r  values. 

The  capabilities  (probability  that  weapon  impacts  within  r  of  the 

target  given  the  system  state  at  the  time  of  release)  are  as  derived  in 

Appendix  B  using  r  =  30  feet  : 

3  „ 
P{hit|A}  =  i     p.  =  .33  -   1/3 
1=1  n 

P{hit|B}  =  1/6 


P{hit|C}  =  0 


Thus,  C1  =  (1/3,1/6,0) 


The  P(hit|B}  is  assumed  to  be  1/6.  The  value  for  the  degraded  mode 
was  not  computed  in  Appendix  B. 
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Solving  for  p  =  A  DC 


p  =  (1/4,5/16,7/16) 


1/3 
1/6 


L°J 


=  13/96 


Therefore,  the  estimated  probability  of  placing  a  weapon  within  30  feet 
of  the  target  with  this  system  is  13/96. 
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APPENDIX  D 
WEAPONEERING  EXAMPLE 

The  weaponeering  usage  of  the  proposed  measurement  and  methodology 
will  be  illustrated  using  the  fictitious  p  calculated  in  Appendices  B 
and  C. 

For  the  computed  p  =  13/96  and  assuming  a  =  .95,  n  can  be  calculated 

as: 

J  u     ln(l-a)  =  ln(.Q5)    =  21  ' 
ln(l-p)     ln(83/96) 

A  graph  similar  to  that  shown  in  Figure  D.l  could  be  used,  vice 
solving  the  above  equation,  to  find  n.  Entering  the  graph  with  p  (called 
the  cumulative  percentage)  and  a,  the  n  could  be  found. 

In  general  usage,  the  weaponeering  requirements  would  be: 

1)  Capability  tables  for  the  applicable  system  and  degradation  modes. 

2)  A  D  vector  for  the  applicable  system. 

3)  Graph  similar  to  Figure  D.l. 

If  the  A  D  were  negligibly  dependent  on  mission  time,  requirements  1  and 

2  could  be  combined  into  a  single  table  of  p  values  for  the  various  values 

of  r  . 
e 

At  the  ship  or  airwing  level,  the  weaponeer  could  use  the  individual 
squadron  capability  tables  or  an  airwing  table  aggregated  over  particular 
systems  in  the  airwing.  (i.e.,  A-7  capability,  F-4  capability,  A-6 
capability).  In  mixed  strikes,  the  ship  level  weaponeer  could  use  a 
ratio  weighting  factor  for  the  p  based  on  the  scheduled  mix.  As  an 
example  consider  a  strike  consisting  of  8  A-7,  4  A-6  and  8  F-4.  The  p 
value  for  the  strike  could  be  p*  =  «4pn  7  +  -^Pn  g  +  -^Pp_4« 
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80rQ  = 


a  =.99 


a  =  .95 
a  =  .90 
a=.75 


Cumulative  Percentage  (c-) 

Figure    D.  I 
Weaponeering  Graph 
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This  paper  is  in  response  to  a  growing  concern  about  the  adequacy  of  present 
measures  and  methodologies  in  depicting  air-to-ground  delivery  system  accuracy. 
An  accuracy  measure,  measurement  methodology  and  performance  guarantee  based  on 
cost-effectiveness  techniques  are  developed.  The  measure  is  based  on  the  frequency 
of  mission  success.  The  methodology  is  based  on  Bayesian  techniques  using  a 
multinomial  distribution  to  represent  the  radial  miss  distance  pattern.  A 
technique  for  using  an  estimate  of  CEP  to  determine  the  prior  parameters  is 
developed.  Actual  and  simulated  impact  data  are  used  to  compare  the  proposed 
methodology  to  historically  accepted  and  other  recently  proposed  techniques. 
The  methods  of  cost-effectiveness  are  applied  to  guaranteeing  an  air-to-ground 
system  in  an  effort  toward  making  costs  more  controllable. 


DD,FN°0RvMe,1473 

S/N    0101 -807-681 1 


(PAGE    1) 


68 


Security  Classification 


A-31408 


Security  Classification 


KEY     WO  ROS 


Bayesian  Analysis 

A-7E 

Cost-Effectiveness  Analysis 

Contract  Guarantee 
Delivery  Accuracy 
Air- to-Ground  Accuracy 
Weapon  System  Analysis 
Bomb  Accuracy 


D,F.T.,1473     'BACK, 


0101 -807-6821 


69 


Security  Classification 


A-  3  I  409 


2  ■>   7  7  T 

23359 


Thes ' s 

M2135 
c.l 


r 


Macke 


7  T8  7« 


Thesis 

M2135 

c.l 


m   ,  134330 

Ma  eke 

A  proposed  method 
for  Hefinino  anH  meas- 
uring weapons  delivery 
system  accuracy. 


thesM2135 

A  proposed  method  for  defining  and  measu 


llllllllllillllllllllllllllllllilllllllilliilllll 

3  2768  000  98207  8 

DUDLEY  KNOX  LIBRARY 


